Papers with MASSIVE dataset
MASSIVE: A 1M-Example Multilingual Natural Language Understanding Dataset with 51 Typologically-Diverse Languages (2023.acl-long)
Copied to clipboard
Jack FitzGerald, Christopher Hench, Charith Peris, Scott Mackie, Kay Rottmann, Ana Sanchez, Aaron Nash, Liam Urbach, Vishesh Kakarala, Richa Singh, Swetha Ranganath, Laurie Crist, Misha Britan, Wouter Leeuwis, Gokhan Tur, Prem Natarajan
| Challenge: | We present the MASSIVE dataset–Multilingual Amazon Slu resource package (SLURP) for Slot-filling, Intent classification, and Virtual assistant evaluation. |
| Approach: | They present a 1M-example dataset of Amazon Slu utterances . they localize the dataset into 50 typologically diverse languages . |
| Outcome: | The proposed model includes exact match accuracy, intent classification accuracy, and slot-filling F1 score. |
MASSIVE-Agents: A Benchmark for Multilingual Function-Calling in 52 Languages (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Using the original dataset, we cleaned up the MASSIVE dataset and reformatted it for evaluation within the Berkeley Function-Calling Leaderboard framework. |
| Approach: | They present a new benchmark for assessing multilingual function calling across 52 languages . they clean the original MASSIVE dataset and reformat it for evaluation . |
| Outcome: | The new benchmark covers 55 functions and 286 arguments in 52 languages. |